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Abstract 

This paper focuses on the rationale and underlying processes of an 
Inter/ intranet tool designed to evaluate oral tests and especially the 
university entrance examination according to the national minimal 
requirements. This exam is taken by up to 100,000 students 
simultaneously. This presentation gives information of the tool design and 
up-to-now results. The authors describe the interfaces, students' reactions 
and test strategies incorporated in the first trials, and present the fun 
operational system. The paper describes some of the experiments of 
image acceptance, attitudes and tool accessibility. The test includes both 
formal and informal language situations, which promotes are likely to be 
found in an international school or daily environment, text correction, and 
develops a feedback model. In the introductory section, the presentation 
describes the tool addressing its strengths and weaknesses and gives 
examples of the different tasks to be done in the exam. Then, it moves to 
the different experiments done in the first semester of 2006 with 
experimental students in the B1 European Framework of Languages level 
and shows their feedback according to a questionnaire. The third section, 
mostly devoted to claim the need of this tool in its current working 
process and design, the researchers will also emphasize the prospective 
research and cases agenda for the project. This project was originally set 
for Erasmus students who needed to be diagnosed to be placed in the 
appropriate levels in foreign languages at the Universidad Politecnica de 
Valencia (Spain). The project was funded by the regional educational 
board with 30,000 Euro and finished in December 2005. Thus, the results 
discussed are tentative and further studies will be necessary. The current 
literature is mostly related to the project and how the new TOEFL oral 
section was developed. 
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Abstract 

This paper focuses on the rationale and underlying processes of an Inter/intranet tool de- 
signed to evaluate oral tests and especially the university entrance examination according 
to the national minimal requirements. This exam is taken by up to 100,000 students simulto- 
neously. This presentotion gives information of the tool design and up-to-now results. The au- 
thors describe the interfaces, students’ reactions and test strategies incorporated in the first 
trials, and present the operational system. The paper describes some of the experiments of 
image acceptance, attitudes and tool accessibility. The test Includes both formal ond infor- 
mal languoge situations, vsThich are likely to be found in an international school or dally envi- 
ronment, text correction, and develops a feedback model. In the introductory section, the 
presentation describes the tool addressing its strengths and weaknesses and gives examples 
of the different tasks to be done in the exam. Then, it moves to the different experimems 
done in the first semester of 2006 with experimental students In the B1 European Framework 
of Languages level and shows their feedback according to a questionnaire. In the third sec- 
tion, rhostly devoted to claim the need of this tool in its current working process and design, 
the researchers will also emphasize the prospective reseorch and cases agenda for the 
project. 

This project was originally set for Erasmus students who needed to be diagnosed to be 
placed in the appropriate levels in foreign languages^ at the Universidad Politecnica de Vc- 
lencia (Spain). The project was funded by the regional educational board with 30,000 Eure 
and finished in December 2005. Thus, the results discussed are tentative and further studies 
will be necessary. The current literature is mostly related to the project and how the nev. 
TOEFL oral section was developed, 

Introduction 

In 2006, the Spanish Ministry of Education will introduce the new educational system that v/ 
change syllabus and, even more important, the initial oge of foreign language learning 
school. The fact that the new syllabus is more communicatively oriented will require 
teaching methodologies but also new ways to measure the students’ progress through the ' 
school period. For a long time, the ministry of education has considered the university c- 
tronce examination (Selectividad) the means to evaluate the students' achievements durirc 
the school years. This common and compulsory examination is held ot the end ol year ' T 
Given its natural importance, teachers and students pay a very especial interest and aacc" 
specific skills and strategies for its preparation. 

A recent publication analyzed English the English test of Selectividad administered in seve^o 
universities in Spain from different perspectives: reliability, consistency, types of items, its ns- 
tory and prospective developments. The two key issues are authenticity of its current forrr — 
{that includes writing, reading and grammar) and whether and how listening and speak'-'-r 
could be included in the test. In fact, a number of researchers in Spain have claimed 
this exam should very its current formot (Garcta Laborda, 2006). However, the cu^’s— 
problem is how to include listening and speaking. 




Creating a computer tool for oral assessment 

A similar problem was faced by Garda Laborda (2004) and Garcfa Laborda & Bejarano 
m05) when onalysing iheir universities’ needs of diagnosing a large number of students d 

Sea onai int^rnd computer software Industry but the interfaces were, as opposed to 
ofherSg platforms, spedfically designed for Internet-based foreign languages ests. Su - 
Sir he SnTresemble the new TOEFL platform however based on cormpletetyj^ 
S philosophy and needs. Besides, the first results of the Vdencian software ^ 
published before the new IBT-TOER started to be operative lost year (Gorcia Laborda, 2004). 

The soeakina module of the Valencian platform of foreign languages (PLWALEX) operates 
Ty uJTg a comdnation of traditional Microsoft software and 

•npcificdlv created for this tool. The student's production is recorded in the studerft s com 
S and r^ord^ continuously and then sent to a database for retneval by the exam 

raters afterwords. Operaftvely, the speaking test has three mam Types of 
, aborda 2005); a pLonal semi-interview, a lecture followed by several questions asked by 
me tSchL ar^d, finally, a description (that can have a formal / acodemic or 'nJorrnaU non- 
-coSc feglster). In the semi-interview, the student watches o video of another student of 
o similar age who asks general questions addressed and corresponding to the proficiency 
■evsl (according to the European Framework) evaluated that are general of ongn ^ 
^Intlfication and then it can be more precise and ask differerft types of questions hane- 
-uire the required proficiency level. In the second part, a teacher will be speaking for two- 
minutes and then asks questions related to the topic, finally the student ^ se® a p\<> 
’ye or academic diagram, will have time to take notes ond will have certain time to 

scribe what has seen. 

Trialling 

PLEVALEX platform hos been designed io be used with a large number of users bemg 
^ultane^^^^^ far. trials have been done with up to 35 studen s simultaneoL^ly^ 
^odifionally the prototype of PLEVALEX is expected to be in use for mitral assessment vj^fth 
sHjdenfs of the Language Center of Unlversidod Politecmca de Valencra by the 
. -“inning of June in the trials, the researchers analysed usability, student eomforl and atti 
; ’-ij^owa^ds the ’testing system. In order to evaluate the usability, 35 students to°k an Eng- 
.''oneral exam of a B1 proficiency level and filled a 25 Item questionnaire. The students 
---rhappy about usability but proposed □ number of changes and, rnore 

. res that the research team ignored and could be interesting for _ 

me comfort one of the researchers met with a group of 10 students and observed the 
m a larae screen with a data projector and discussed the different sections but od- 
:i* -nal information was obtained through this method. As a conclusion of the three firet set 
-f mojs it was concluded that the platform can begin to be experimentally 

students are happy with the current format. Other minor comments lead to slight 

zrer^QBs in format. 



Implications for the University Entrance Examination and conclusions 



The results obtained so far have created the expectations that a similar web tool could be 
applicable to larger numbers of students. However, reaching 100,000 students simultoneously 
seems to be far beyond this project's scope. Although the use of synchronous servers could 
reproduce the exam to a definitely lorge number of students, streoming and communication 
seem to be two problems thot given the current potential of the net cannot be solved today 
but we hope that in just a tew years these problems could be solved. 

The consequences of this Spanish education and, more precisely, the University Entrance 
Examination would benefit greatly from the introduction of this platform os a testing tool be- 
cause it could include items of languoge skills that, although recognized as necessary, by 
institutions, researchers and general teachers have not been included yet. To provide 
Spanish high schools with solutions to the technical problems this study suggests the use of 
parallel servers for large quantity of this high stakes exam and further research com com- 
plement the current ope. The results so far a very promising and hopefully the universities will 
a very valuable tool for language assessment and diagnose at low cost which will certainly 
benefit the Institutions as well as the individuals. 
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